1. Field of the Invention
The present invention relates to the testing of electronic systems, and, more particularly, to a method and apparatus for automatically generating tests for a digital system.
2. Description of the Related Art
Advances in digital electronic technology allows the development of increasingly complex digital electronic systems that include a combination of hardware and software components. These systems must be tested, or functionally verified, to ensure that they operate correctly under as many conditions as possible. It is widely accepted that the complexity of functional verification for a system increases as a square of system complexity. Currently proper verification of a new system is one of the biggest challenges when developing a new product, consuming 50% or more of development resources.
The central problem in functional verification is trying to cover every possible condition, or scenario, that the design may encounter during operation. Functional verification should cover (a) each individual design function, (b) each architectural component of the design, (c) interactions between design functions (and preferably all such interactions) (d) interactions between architectural components (again, preferably all such interactions), and finally (e) interactions between design functions and architectural components (again, preferably all such interactions).
Formal verification techniques attempt to mathematically prove correctness of a design without requiring any tests, but these techniques are generally able to handle designs of limited complexity. What is required is a collection of thorough functional tests that can be run in either simulation or on the fabricated product to find any potential flaws in the design. Furthermore, in order to support development of large, complex designs, these tests should support hierarchical verification: in additional to verifying the whole system, it should be possible to create tests for part or all of any module or collection of modules that make up the system.
Modern digital electronic systems are too complex to analyze the entire system at once when considering the possible cases that require testing. Instead it is necessary to carefully analyze both the functional specification(s) and the design architecture for the system to consider all possible test cases. Both the functional specification and the design architecture should be broken into a set of component modules that are small enough to analyze as a unit. Each module can then be considered independently in order to define the set of tests that could be run on that module. The linkages or interactions between modules can then be analyzed separately.
Such analysis, especially when considering all possible tests for all module, will lead to a very large number of possible test cases. Limitations of time and equipment make it possible to run only a small fraction of these tests. A method is required to efficiently capture the entire space of possible tests, and then select a random distribution of tests to run from that space. It should be possible to combine information about possible tests for multiple modules to create tests that verify both individual modules and the interactions between those modules. A method is needed to control the scope and distribution of the generated tests to ensure sufficient testing of all parts of the system at different levels of hierarchy.
The required format for tests may be very different for different stages of design development and at various levels of verification hierarchy. For example, the style of tests needed when running in simulation may be very different from the style of diagnostic tests required when running on a fabricated system. Elements of test style may include how the test is applied to the design, how expected results are created, and how the set of tests is analyzed to measure how much of the design has been tested.
A number functional verification tools are currently available. However, none of these tools provide the verification engineer with any mechanism to analyze a complex design in terms of its component modules, instead requiring the engineer to think about testing the entire system as a whole. No mechanism is provided for capturing information about a complex set of possible tests, or for selecting a random distribution of tests from within that set.
Some tools provide engineers with a mechanism for generating random stimulus for specific inputs to the system. These solutions may test part of the module which receives that input, but do not consider the possible interactions this module may have with the rest of the system—in other words, only a small subset of the possible tests are covered. Other tools provide tests for a specific design component (e.g., a microprocessor), but do not lend themselves to be extensible by the verification engineer to test the rest of the system—again, only a portion of the possible set of tests are being covered.
It is difficult to use these tools to generate tests for use at different levels of verification hierarchy, because (a) the tools lack the flexibility to properly cover all (or even substantially all) possible tests at each level of the hierarchy, and (b) the tools do not support the different test formats that may be required at various levels of verification hierarchy.
What is therefore needed is an apparatus and method for functional verification that overcomes the problems of prior art.